9.1 The Convolutional Operation¶
Convolution is an operation on two function of a real valued argument.
The conv operation is usually denoted with an asterisk
In the case of indiscrete value
\[s(t) = (x * w)(t) = \int x(a)w(t - a)da\]
In the case of dsicrete value
\[s(t) = (x * w)(t) = \sum_{a=-\infty}^{\infty} x(a)w(t - a)\]
Note: w needs to be 0 for all negative arguments
- x: input
- w: kernel
- s: feature map
For 2D input and 2D kernel:
\[S(i, j) = (I * K)(i, j) = \sum_m \sum_n I(m, n)K(i- m, j - n)\]
It is commutative, so we can also right
\[S(i, j) = (K * I)(i, j) = \sum_m \sum_n I(i-m, j-n)K(m, n)\]
Cross-coorelation:
\[S(i, j) = (I * K)(i, j) = \sum_m \sum_n I(i + m, j + n)K(m, n)\]
Many machine learning libaries implement cross correlations but call it covolution. Check reference for the difference.
Any Neural Network algorithm that work with matrix multiplication and does not depend on specific properties of the matrix structure should work with convolution, without requiring any further changes to the neural network.